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ABSTRACT 


An efficient data compression system is presented for satellite 
pictures and two grey level pictures derived from satellite pictures. 

The compression techniques take advantage of the correlation between 
adjacent picture elements. 

Several source coding methods are investigated. Double delta 
coding is presented and shown to be the most efficient. Both predictive 
differential quantizing technique and double delta coding can be signi- 
ficantly improved by applying a background skipping technique. 

We have investigated several classes of codes to represent the 
derived data resulting from source coding techniques. An extension 
code is constructed. This code requires very little storage space and 
operates efficiently. Simulation results are presented for various 
coding schemes and source codes, 

A method is presented to measure the complexity of pictures. Data 
compression ratios can be predicted from the measurement of picture 
complexities. 

A compressed data format is designed to pack the encoded derived 
data into a more compact form for better data compression. This format 
also protects the data so that the number of objects in the picture can 
be decoded correctly. 


Three efficient coding techniques for satellite pictures are in- 
vestigated. Ail three can be' used in a nearly real time, multi-rate 
data transmission system. One of them is a modified DPCM system where 
various codes and quantization schemes are used. A hybrid coding com- 
bining DPCM and a transformation technique is also presented. This 
technique minimized the rms error. The linear interpolation technique 
is also examined. The advantage of this technique is its low cost of 
Implementation. Performance comparisons among these techniques are 


CHAPTER 1 


INTRODUCTION AND LITERATURE SURVEY 

The discrete multi spectral scanner data are sampled and quantized 
into a fixed number of bits for each picture element. This is referred to 
as a satellite picture, which usually requires 6 to 8 binary digits per 
picture element (pixel). Some two-level and multi-level pictures were 
obtained from picture classification on satellite pictures. 

Due to the redundancy In the pictures, various data compression tech- 
niques were developed to reduce the data bulk to be transmitted. In this 
chapter, we make a technical review of the literature, followed by the in- 
troduction to the objective of this thesis. 

1.1 Technical Review of the Literature 

People have attempted to acquire information from measurements made 
by a Satellite at a distance above the earth. This major area of research 
provides a means to predict and control man's environment [1,2]. The 
data is collected by a sensor mounted on the aerospace platform. The re- 
flected energy in certain regions of the electromagnetic spectrum are 
collected and digitized to form multlspectral data [3]. 

To extract information from the multi spectra* data, and to interpret 
it as certain useful objects, one has to relate the properties of the 
object with the measurements on the spectral, spatial, or temporal 


domains 
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Basical 1y» there are two approaches toward the classification of 
digitized imagery; namely, statistical and 1 ingulstic approaches. More 
research has been done on the statistical approach than on the linguistic 
approach. The advantage of the statistical approach Is its generality, 
although complete statistical information of the objects is impossible. 

Most of linguistic methods transform input pictures into line drawings or 
regions [A, 5 ], and apply natural or artificial language processing. 
Haralick, Shanmugan and Dinstein [ 6 ] investigated the idea of using tex- 
tural features for image classification where the spatial relationship 
between picture elements was used to train the classifier [ 7 l. 

Wilkins [ 8 ] and Gupta [ 9 ] attempted to extend the blob algorithm to 
find the natural boundary. But two picture elements belonging to the 
same class may belong to different populations in certain channels. Thus, 
blob algorithm may give misclassi f ication in this circumstance. 

A subset of mul ti spectral measurements Is found to contain most of the 
information which would provide excellent classification results. Feature 
selection techniques can be used to find the best combination of channels 
from the original N channels [10,11]. 

LARS, Purdue University [12], developed a multispectral data analysis 
system. Based on the assumption of Gaussian distributed data, the cluster- 
ing analysis [13»1^»15]» was used to decompose the non-Gausslan density 
function into several Gaussian density clusters. This will provide the 
statistics for training classes corresponding to each cluster and can be 
repeated to refine the training samples [ 16 ]. 

The table look-up algorithm for picture classification [17] is an im- 
proved technique. This approach has reduced the processing time signifi- 
cantly compared with the approach by LARS on the same computer. 
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Many research projects have been conducted on the data compression 
for binary and continuous- tone pictures. The process of scanning reduces 
a picture from a two-dimensional array of pixels to a one-dimensional se- 
quence of pixels, which can be shown to be a first order Markov process 
[18,19]. Later Habib i [20] and others [21,22] extended the model to a 
two-dimensional random field. 

Two-level and multi-level pictures are characterized by sharp edges 
due to limited number of grey levels. Thus, they often can be compressed 
with no degradation. Run length coding [23] and the scheme suggested by 
Huang [24] were developed to compress the binary pictures such as text 
and weather maps where the figures are mostly line drawings. Run length 
coding takes into account only horizontal correlations in the scan line. 
The two-dimensional run length coding known as predictive differential 
quantization (PDQ) [25] was developed to take into account both horizontal 
and vertical correlations. 

Bit-plane coding [26,27] was used to encode the multi-level pictures. 
This algorithm transfers a n-bit per pel picture into n pictures of 1 bit 
per pel, followed by the run length coding (RLC), et.al.. Thus, the same 
coding process of RLC was repeated n times. The data compression achieved 
is not significantly better than other schemes. Indeed, it really depends 
on the complexity of objects. 

Because of the complexity of the continuous-tone pictures, any error 
free coding technique can not provide significant data compression. Those 
coding schemes which result in certain amount of distortion had been in- 
vestigated. Kretzmer [28] and Harrison [29] exploited the correlation in 
the video signal in order to reduce the average power required to trans- 
mit it, and found that a substantial reduction in the average power 


C0u1d be obtained using just the previous element, little further re- 
duction was possible using more of the adjacent elements on the same and 
preceding lines. Schrelber [30] measured the second and third order 
statistics of a scan line and found considerable correlations between 
adjacent elements. 

Plecewlse-1 Inear approximation for Interpolative coding was investi- 
gated by Youngblood [31]. By dividing the picture Into small blocks, 
Cunningham [32] represented the grey level of each block by Its average 
griey level. Graham [33] regarded all points with the gradient higher than 
a threshold as edges. By transmitting changes In contour directions, 
gradient [3^] directions, and magnitude using Huffman codes [35] » pictures 
can be reconstructed at the receiving end. Later, contour coding was 
used with the blob algorithm [8,9]. However, Its shortcomings are the 
large memory soace for storing the whole frame and relative complexity of 
the IP and T algorithms. 

DPCM Is based on the Invention of Cutler [36]. The differences be- 
tween the prediction and the signal are quantized Into one of m grey 
levels and a code Is transm* Lted to Indicate which of the m levels occurred. 
The quantization scale could be either uniform or nonuniform. Theoreti- 
cal studies of the nonuniform quantizer had led to Max's [37] criteria 
for minimizing the rms quantization distortion. The most desirable 
characteristics of DPCM Is the ease of design and the speed of the oper- 
ation. The limitation of this system Is Its sensitivity to the picture 
statistics and the propagation of the channel error. 

The transform coding [38-41] was designed to create a domain In 
which the data is uncorrelated and the signal energy is compacted 
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Into a small number of components. The typical transform coding includes: 
Fourier, Hadamard, Hotelling and linear transform. The advantage of the 
transform coding Is its superior coding performance at lower bit rates 
and the degradation is less objectionable to a human viewer, in addition 
to Its less sensitivity to the data statistics and channel errors. But 
this scheme usually requires a large memory space during the processing. 

An extensive survey of published papers and reports dealing with 
data compression, picture properties, picture coding and transmission 
can be found In Wilkins and Wlntz [^2], Huang, et.al. [^3l » 3nd Connor, 
et.al. [ 45 ]. 


After the redundancy In the digitized picture is removed by some 
cpding scheme, efficient source codes are used to encode the derived 
data of the coding schemes such that they can be transmitted over the 
communication channel. 

There are two types of codes,- fixed length codes and variable 
length codes. The natural code and Gray code [46] are two typical fixed 
length codes. In the natural code, the numbers are represented by their 
binary representations. Gray code is an inverted binary code. The ad- 
vantage of the fixed length code is that the code length of each code 
word remains the same which makes the design of the data format a lot 
easier. It also has a drawback that the redundancy still exists in the 
codewords due to nonuniform distribution of the data. 

The variable length codes are designed to map shorter code words to 


the data with higher probability of occurrence, and long code words to 
rarely occurring data. The Huffman code [35] is known to be the optimal 
variable length code, but It Is not practical for coding purposes. 
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Huang, et.al. developed a code to encode the run lengths for run length 
coding of binary pictures. The prefix code [^7l is another variable 
length code, but it is not suitable to encode the run lengths and the 
data where their values could be an integer. 

In general, certain codes may be good only for certain type of data. 
To match the code words with the statistics of the data is the key in de- 
signing the efficient source codes. For more theoretical treatment of 
the coding aspects, Stiff ler [4?] and Gal lager [48] are good references. 

1.2 The Objective and Contributions 
The objective of this research Is to investigate an efficient data 
compression system for both continuous tone satellite pictures and those 
pjictures with two or more grey levels. A block diagram of the data com- 
pression system is shown in Figure 1.1. The system is divided into two 
subsystems for compression and decompression purposes, 

A digitized picture is processed by the redundancy remover to achieve 
the first level of data compression. The output of the redundancy remover 
is calied the derived data. Some efficient source codes are used to en- 
code the derived data and remove the redundancy in the data due to their 
nonuniform distributions. This processing is also referred as the second 
level of compression. In general, the output of the encoder for each 
scan line is a variable length bit string. Slock packing will pack the 
bit strings into fixed length blocks such that data transmission can be 
a:chieved at a constant bit rate. The received messages can be decoded in 
reverse order as shown in the figure to obtain the reconstructed picture. 

Chapter 2 briefly describes the picture classification (or pattern 
recognition) algorithms which convert the multispectral satellite pictures 
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ijnto two-level and multi-level pictures [1»9]. Run length coding (RLC) and 
predictive differential quantizing (PDQ) were originally developed for data 
compression on the binary pictures. We have modified the PDQ to ease the 
design of an efficient compressed data format. An improved data compres- 
sion technique is introduced as the double delta coding (DDC). DDC is 
shown to be more efficient than PDQ. We find that the complexity of the 
figures in a picture plays an Important role in the data compression ratio. 

Efficient source codes are developed to encode the derived data and 
achieve the second level of data compression. In Chapter 3, we present 
the algorithm to construct the extension code which combines the prefix 
code and folding technique [50]. Three classes of the non-prefix codes 
are also investigated. The code words of the non-prefix code consist of 
blocks of fixed length digits. The first bit of each block is the identi- 
fication bit, such that the start and end of the code word can be recog- 
nized. The extension code and non-prefix code are designed to match the 
source statistics. 

For a multi-ievei or two-level picture, certain grey levels are more 
likely to occur than others, and we define this grey level as the majority 
class. Regarding the majority class as the background class, background 
skipping will enable us to process those figures represented by the 
minority classes. Thus, the processing time required will be reduced 
significantly in addition to the improvement of the data compression ratio, 
in Chapter i|, we also include the computer simulation results by applying 
the coding techniques and codes discussed in the previous chapters. An 
efficient compressed data format is designed to pack the coded messages 
sjuch that they can be decoded correctly at the receiving end. A mathe- 
matical model for the derived data is set up, from which an upperbound for 
the entropy is defined. 


Chapter 5 deals with the compression techniques which provide the 
data compression ratios of four and six for the satellite Images. In this 
case, any error free coding scheme can not guarantee the desired data 
compression. Thus, the distortion resulting from the data compression Is 
Inevitable. An error criterion Is defined to serve as the basis In evalu- 
ating the performance of the compression schemes. Three different schemes 
are developed and simulated. The overall performance has to take into 
account both the processing time and implementation complexity. 



CHAPTER 2 


DATA COMPRESSION ON SATELLITE IMAGES AFTER CLASSIFICATION 


In this chapter we will investigate some efficient picture coding 
techniques on two-level and multi-level pictures resulting from image 
glassification on mul tispectral satellite images. In section 2,2, we 
briefly describe the picture classification. Then, run length coding and 
modified predictive differential quantizing are investigated in section 
2,3 and 2, A, Finally, an Improved version of a coding technique called 
double delta coding is introduced in section 2,5, This chapter is closed 
by a section which investigates the complexity of the objects contained 
in the pictures, and its Influence on the selection of the appropriate 
coding technique. 


2,1 Benefits of Efficient Image Coding 
A digitized image is a two-dimensional array of picture elements re- 
suHing from the quantization of sampled data. Each picture element is 
assigned one of the 2*" grey levels when a m-bit quantizer is used, where 
m is an integer. When a multispectral image is involved, the third di- 
mension is the spectral bands. For example, the LANDSAT image consists of 
four spectral bands. That is to say, there are four pictures of the same 
scene. 


For continuous tone pictures, usually a value of m between 6 and 8 
is required to obtain a satisfactory picture representation. The size of 
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the mul tl spectral satellite images is about 2350 by 3260. There are usu- 
ally four or more pictures for the same scene. Thus, we need about 120 
frames to cover the complete information for our research interest. It is 
rather difficult and time consuming to handle such a huge amount of data. 
Fortunately, there is a lot of redundancy in the pictorial data, since the 
entropy of the picture is less than the bits used to represent the picture. 
The redundancy reduction techniques developed can be divided into two 
categories. One of them is for continuous tone pictures which are 
characterized by smooth edges and fine variations between samples. The 
data compression schemes for continuous tone pictures will be discussed 
later in Chapter 5* The other is for binary and multi-level pictures 
which are characterized by sharp edges due to the limited number of grey 
levels. Therefore, this type of pictures should be compressed without 
any degradation. 

The two-level and multi-level pictures with which we are concerned 
were obtained from image classification on multi spectral satellite images. 

typical examples of this type of data are water/non-water map (two-levels), 

I 

land use map (five levels), and snow cover map (three levels). To trans- 
mit this type of data accurately within a short period of time will re- 
quire a very efficient data compression method. 

To evaluate the performance of the data compression techniques, many 
factors will have to be considered, for example, the computational efforts 
to do the encoding and decoding, the data compression ratios, and the 
effect of transmission errors. A sophisticated coding scheme may pro- 
vide an attractive data compression ratio, but it may also require more 
computation time both for encoding and decoding. Our goal is to 
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Investigate those coding techniques which will require minimum equipment 
cofnplexity and operation efforts. Based on this principle, we wl i 1 in- 
vestigate some promising candidates in latef sections. 

2.2 information Extraction by Image Classification 

For each ground resolution element, the sensor on board the satellite 
generates a set of n measurements corresponding to n spectral channels. 

We can regard tiiese n measurements as a point In n-dimensional Euclidean 
space (or measurement space). To reduce the size of the data bulk and 
computational efforts, a subset of n components of this measurement space 
Is selected, and used in the picture classification. Feature selection 
techniques select four out of n channels as a best subset which couid 
provide good classification results. 

There are several classification algorithms developed for computer 
processing. Among them, the Gaussian maximum likelihood classifier [12] 
and the table look-up method [17l are well known. In the Gaussian classi- 
fication scheme, the data is assumed to be multivariate Gaussian distri- 
buted. If the original data is non-Gaussian distributed, clustering [51 1 
would provide a mechanism for dividing the training class into approxi- 
mately Gaussian subclasses. Then, these subclasses can be used as the 
refined training samples to train the classifier [7l. 

Based on the maximum likelihood decision rule [52], [53l, each pic- 
ture element is classified into an appropriate class. A flow chart In 
Figure 2.1 shows the procedures of the Gaussian maximum likelihood classi- 
fication. 

The table icok-up method is a powerful classification technique 
which substantially reduces the processing time compared with the 













Gaussian classifier on the same computer. This method makes it possible 
to process muitispectrai data on the minicomputer without additional 
special-purpose hardware. The classification procedure for each picture 
element involves a sequential, tree-structure search in which each 
measurement is tested to determine whether or not it is inside the region 
of measurement space assigned to the hypothesized class. The partition- 
ing of measurement space is accomplished by the program which prepares the 
table. At each stage, the permissible range is a function of the class 
and the values in the preceding channels; for example, with a 4-channel 
satellite picture, the picture element is represented by the measurement 
vector » (XpX 2 ,X^,X|^) , Let X^j^ and X^^^ be the minimum and maximum 
values that X^ can have and still be inside that region of a four-dimen- 
sional measurement space associated with class C. If X^^ the 

hypothesis C holds, otherwise a new hypothesis is formed and tested. 
and X^j^ are functions of C, Xp X 2 , X^. If at any stage, an input data 
value is outside its permissible range, a new class hypothesis is formed. 
Testing continues in this way until the desired class is found such that 
all four data values are within their permissible ranges. 

Vte have classified the LANDSAT pictures, taken in normal and flood- 
ing periods, into a reservoir map, river map, snow cover map, and land 

j 

use map. These maps will be used in the picture coding simulation later. 
Classification results are presented in Figures 2.2 through 2.9. It is 
Worthwhile to mention that classifying the picture into water and non- 
water using only two channels did provide comparable results with that 
using all four channels. This is verified by Figures 2.2(b) and 2.2(c). 
Furthermore, the processing time is reduced significantly. 



Figure 2,3 Thresholding and Classification Results 

(a) Thresholding Display of Figure 2.2a 
Non-Flooded Reservoir 

(b) Classification Result of Flooded 
Re servo I r 












2.8 Snow Cover Map 

(a) Original (Channel 2) 

(b) Classification Resul 
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Since water reflects little of the light in channel of the LANDSAT 
picture, it appears as a significantly dark area. An easy way to classify 
the picture into water and non-water is to threshold the picture (channel 
4) with an appropriate threshold value, instead of running the classifi- 
cation programs mentioned above. The CPU time used by the thresholding 
technique is minimized. Figure 2.3(a) shows a thresholded picture result- 
ing in a reservoir map. In Figure 2.6(a), the interstate highway, runway 
of the airport, and buildings in the residential areas which are made of 
concrete, reflect most of the light in channel 1, and therefore appear as 
the bright areas. 

The color picture shown in Figure 2,5 is a simulated infrared (IR) 
picture using channel 1, 2, and The forest is shown as red in IR pic- 
ture. This picture is classified into five classes: concrete/highway, 

water, soil, vegetation, and forest. Each class is identified with a 
color as shown in Figure 2.7(a). The colors in the color chart as shown 
In Figure 2.7(b) were used to represent the classification results. 

Figure 2.8(a) was taken in March of 197^* which provided the information 
about the snow cover on the ground. It is classified into three classes; 
water, non-water, snow. In Figure 2.8(b), these three classes were 
represented by dark, grey, and white respectively. A color coded classi- 
fication result of Figure 2.8(a) Is shown In Figure 2.9(b). 

V . . 

2.3 Run Length Coding (RLC) 

Those pictures obtained by the picture classification of mul tispectral 
Images consist of sequences of class numbers, where each class number 
corresponds to one of the grey levels. For example. In the binary water- 
surface extent map, grey levels 0 and 1 represent water and non-water 
respectively. 
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The neighboring picture elements of a binary or multi-level picture 
are highly correlated. In other words, the chances for the neighboring 
picture elements to be of the same grey level are very good. Instead of 
transmitting the class numbers point by point, the run lengths of each 
class and the class number are transmitted. Thus, this coding scheme is 
named run length coding. At the receiving end, simply repeat the class 
number as specified by the run lengths to reconstruct the picture. 

There are two parameters resulting from run length coding: the class 

number and run length. Thus, the derived data will be pairs of integers 
(C,L), where C stands for the class number, L is the run length. After 
the application of run length coding, the picture can be represented by 
a set of sequences: 


) » (^ 2 * ^ 2 ^ 


The only constraint is that the successive class number C| should not be 
the same. 

Suppose that the length of a run depends only on the class number of 
this run, and the class number statistically depends on the class number 
of the previous run; then, the entropy of a typical sequence of n runs 
can be represented by 


H (C^ , L| , C2, L2, . . *C^ , L^) 


n 

E 

K=1 




(2.1) 


The entropy of a sequence of m^ runs of class 1, m2 
etc. is 

n n 


E m„ H(L„|C„) + E 
K«1 ■ K» 


■"K 


runs of class 2, 


( 2 . 2 ) 


By the definition of the entropy: 
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where P(l) is the probability of the run length. The average number of 
the picture elements in the above sequence is 
n 


Z m|^E(L|C|^) 


K=1 


where E(L|Cj^) is the average run length for the class K. 
Therefore, the entropy per picture element is 


h(L) « 




^ m^ E(l|C^) 


( 2 . 1 .) 


This is the lower bound on the number of bits per picture element re- 
quired by a code provided that the successive runs are independent. 


Z,k Predictive Differential Quantizing (PDQ) 

Predictive differential quantizing (PDQ) is an extension of the run 
length coding. In run length coding, the correlations among the picture 
elements along the scan lines are taken into account. This is generally 
referred to as one-dimensional R.L.C.. PDQ takes into account both hori- 
zontal and vertical correlations. 

The main idea of PDQ is to transfer the differences between corres- 
ponding run lengths of successive scan lines. The symbols L' and L" as 
shown in Figure 2.10 are transmitted, along with signals of new starts 
and merges. A** indicates the change of location, while A"" tells how the 
width changes. Vfe modified PDQ to include new starts and merges as 
special cases of A', such that the new start signal will be coded by its 
starting column number, and the merge signal is encoded by a termination 
code. This also makes the format design easier. In our simulations, the 
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Figure 2.10 Predictive Coding 

merge signal (termination code) is represented by the number of samples, 
n, of a scan line plus one (i.e. n-<-l). More details about the codes will 
be given in Chapters 3 and k. We defined the process of converting the 
picture into sequences of (A**, A"") pairs as the first level of compression. 
And the process of encoding the A's with appropriate code words is the 
second level of compression. 

Huang [25] concluded in his paper that PDQ out performs both run 
1 en 9 th coding and block coding when it is used on weather maps, but it is 
inferior to run length coding when used on printed material. We have 
applied PDQ to our two-level and multi-level pictures. From the simula- 
tion results, it showed that the complexity of the figures is a major 
factor in the data compression ratio achieved. - 

2.5 Doubie Delta Coding (DDC) 

As shown in Figure 2,10, instead of transmitting A' and A'", we 
propose to transmit A"* and L'" such that both A" and A'"*'* indicate 
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the changes of the edges. Again, the new start and merge signals arc re- 
garded as special cases of A", In these cases, L'** is used to represent 
the width of starting or merging figure on that scan line. 

Let us consider the following figure 


Scan line i 




Scan line 1+1 



Figure 2.11 Double Delta Coding 
and define the A's in the strict sense. 


4' -Sj-S, 
4 " » 


4'" - S^-S^ 


(2.5) 


Assuming that straight lines could be used to approach end points 
Sj, %y and S2» S|^ respectively. As the strict sense of the definitions 
of A's in equation (2.5) is concerned, we can assume that the probability 
distribution functions of A" and A'*'"' are approximately the same from 
their definitions. Thus, we have H(A") = HCa'"'), However, we have used 
the general definition of A** in our computer simulation for convenience. 
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The entropy for new starts and merges were included in HCa"), H(A") is, 
therefore, higher than 

The probability density functions of A's are shown in Figures 4.3 
through 4.6, We see that there is always a highest spike dominating 
over the rest of them in the distribution of h'" , However, there is 
no such dominant spike in the distribution of L" , From the properties 
of the entropy function [48], we conclude that the entropy of L'" is 
less than that of A"" (i.e. H(A'’‘'‘*) < H(A'"’)), This conclusion is veri- 
fied by our computer simulation results as shown in Table 4.6. 

It Is more efficient to transmit A" and h'" than L' and A'"', not 
only because the number of digits required to be transmitted is reduced, 
blit also from the practical point of view. This is the advantage of the 
double delta coding over predictive differential quantizing. The computer 
simulation results will be shown in Chapter 4. 


2.6 The Complexity of the Figures 

Most of the source coding techniques are data dependent. In other 
words, certain coding schemes may be good for certain pictures, but It is 
less efficient for other pictures because of the design of the coding 
algorithm. 

For those pictures shown in section 2.2 which will be used in the 
computer simulations, the complexity of the picture Is a major factor In 
the performance of the coding scheme. 

Let us define the complexity of the figure i in the picture as 


follows: 


where Aj Is the area of the figure I and Pj Is the perimeter of the figure 

I. 

Among all the figures, the circle is the simplest figure wi’th C. = 0. 
Thus, the complexity for the circle is the minimum for all figures. On 
the other hand, a line can be regarded as an extreme case of a complicated 
figure with Infinite perimeter and infinitesimal area, thus Its complexity 
Is 1. The complexity of any figure should He between 0 and 1. 

Since the area and perimeter of a figure can be obtained as a by- 
product in the processing of run length coding or predictive coding (PDO 
or DDC), there will not be much extra computer time required to determine 
the complexity of the figures. 

The complexity of the whole picture, C, can be defined as the sum of 
the complexity of each individual figure In the picture. That Is, 
n 

C = Z C. (2.7) 

I«1 ' 

From our computer simulation, we found that run length coding outperformed 
PDQ and DDC for the pictures with high complexity. For example, the 
flooded river map Is a picture with relatively high complexity. On the 
other hand, PDQ and DDC provide better data compression ratios for the 
pictures with little complexity. The river and reservoir maps taken In 
the normal season are typical examples. For the multi-level pictures like 
the snow cover map and the land use map, run length coding would be more 
efficient because of the picture complexity. 

In Figure 2,12, we show the relationship between the compression 
ratio and the picture complexity for various coding schemes which were 
used In our simulations. Thus, a threshold, T, for the picture complexity 


"■JO 9 12 15 18 21 2^ 27 30 

DATA COMPRESSION RATIO 

■ Run length coding 

Predictive differential quantizing 
****** Double delta coding 

Figure 2,12. Data Compression Ratio vs Picture Complexity 
for Simulated Pictures 
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can be chosen properly such that whenever C^T, the RLC will be used as 
the redundancy remover, whereas PDQ or DDC Is chosen if C< T. 
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CHAPTER 3 

EFFICIENT SOURCE CODES 


The first level of redundancy removing was Investigated In Chapter 2 
In which most of the coding schemes take Into account the high correlations 
of the neighboring picture elements. 

Through the redundancy remover, we convert the Input data (digitized 
image) into sequences of run lengths or Index pairs, which are defined as 
the derived data. In this chapter, we will Investigate the properties of 
the derived data. Several classes of code words were designed to match 
the statistics of the derived data, 

3.1 Source Codes to Match the Data Statistics 
From the probability density functions of the runs and A's which are 
shown In Figure k.3 through k.6, we see that the statistics of the de- 
rived data of the computer simulated pictures has a characteristic that 
the data falls in the range -31 to 31 with the probability of 0.6. 
Furthermore, the probability density functions can be approximated by 
exponential and symmetric functions. 

The optimal code for this type of data is to assign shorter code 
words to the data with higher probability of occurrence, and longer code 
words to the data with less probability of occurrence. The Huffman code 
C35] is the optimal code for nonuniformly distributed data, but the con- 
struction of this code is too complicated to be Implemented in practical 
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operations. Thus, the code words which we will investigate for the de- 
rived data are sub-optimal codes. 

3.2 Properties of the Prefix Code 

A sequence^ p, of binary symbols is called a prefix of m if o)=ps. A 
class of codes is said to have the prefix property if no code word is a 
prefix of any other code word. Thus, if a class of codes has the prefix 
property, each code word can be uniquely decoded as soon as it is received 
In its entirety. However, the classes of codes which will be discussed 
later in section 3.4, still can be uniquely decoded even though they are i 

not prefix codes. I 

To demonstrate the importance of the prefix property, let us consider 
a class of codes C = (11, 010, 100, 110, 101, 0011} which does not possess 

i 

the prefix property. Suppose that a sequence of symbols S = 110101010101 
was received. S can be decoded into 11, 010, 101, 0101, or 110, 101, 010, 

101. That is to say that S cannot be decoded uniquely. Therefore, a I 

I 

uniquely decodable code has to be a prefix code or use a special design of | 

Identification bit in each code block, such as the code in section 3.4. 

Prefix codes can be constructed through the tree structure [54] which 
Is the easiest way to form the code. The prefix code is utilized as the 
first step of the construction of extension codes in section 3.3. 

3.3 Algorithm CT - to Construct the Extension Code 
The first step of the construction of Extension Codes is to form a 
class of prefix codes by the tree structure as mentioned in section 3.2. 

The unique decodab! 1 ity of a class of codes can be verified by a method 
suggested and proved by Sardinas and Patterson [551. 

i 
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For those pictures we have simulated, the norma) operating range of 
the derived data Is about 700 Integers. To encode the derived data with 
prefix codes will require a large amount of code words which make the 
construction of prefix codes Impractical for coding purposes due to the 
code length being Increased too fast. 

Our algorithm Is to construct a class of prefix codes which consist 
of a few code words. Applying the extension blocks to some of the prefix 
codes, we can extend the prefix code with a limited number of code words 
to encode a large number of source messages (I.e. derived data). And the 
coded sequence can be decoded without ambiguity. 

It Is clear that the storage space required to store those prefix 
codes at the encoder and decoder will be very small. The Implementation 
of the extension code can be done with very little complexity. It is 
easier to explain the algorithm CT with an example. 

Suppose that the source messages consist of Integers ranging from 1 
to 560. We construct a code dictionary C = { 00, 111, 1100, 1101, 1000, 
1001, 1010, 1011, 0100, 0101, 0110, 0111}. The code C Is a class of pre- 
fix codes. Since any sequence of words from C beginning with any suffix 
of any word In C can make the decoder automatically come into correct 
synchronism, the code dictionary Is a completely self-synchronizing 
dictionary. 

Again, we assume that the small Integers possess higher probability 
of occurrence. Figure 3*1 shows the mapping of the code words to the 
source messages. Of course, the number of code words In C can be adjusted 
based on the number of source messages (or derived data). This code re- 
quires very little implementation complexity and very small storage space 
to store the code words. All of these features make the extension code a 
practical source code. 
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Code 

Vtords 

00 

ill 

0101 

1101 

1000 

0111 

0110 

1011 

0100 

1100 

1010 

1001 


Mapping To 
Source Messages 

1 

2 

3 

h 

5 

6 

7 

8 
9 

1 block of extension (3 bits) 

2 blocks of extension (6 bits) 

3 blocks of extension (9 bits) 


0101 + ( I i ( will cover 10 to 17 (16 code v/ords) 

0110 + I I 1 ~1 I i 1 I will cover 18 to 81 (6^ code words) 

Oni + I I I I i I I I I f i I will cover 82 to 593 (512 

code words) 

In other v/ords there are 2^ possible ways to 
fill the biock/blocks. 




Figure 3.1 Extension Code for Integers 1 through 593 
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The code words shown fn Figure 3*1 can be used to encode the run 
lengths. To encode the derived data from predictive coding, where the 
Integers are ranged from -275 to 1060, another mapping will be used, as 
shown In Figure 3*2. For L' , we assign the shortest code word to initiate 
3; figure termination. The code words 0100, 1100 followed by one or two 
blocks of extensions are assigned to encode the positive numbers such 
that 0100 plus 5 bits of natural codes will represent the integers k 
through 35» and 1100 plus 10 bits of natural codes are used to encode the 
integers 36 th'^ough 1060. Similarly, code words 1010 and 1001 are extend- 
ed to encode -1^ through -19 and -20 through -275 respectively. Note that 
the number of bits in the extension blocks can be adjusted such that the 
extension code can cover the whole range of the derived data. 

3.^ Non-Prefix Code 

We have said that the uniquely decodable code has to possess the pre- 
fix property. In this section, we will investigate a class of codes 
which is not a prefix code. The structure of this code is different from 
that of the prefix code. In order to uniquely identify each code word, 
the code word is divided into several fixed length blocks of two bitSi 
The first bit of each block Is the identification bit followed by an 
information bit. All the identification bits within a code word have to 
remain the same so that it can be recognized and decoded correctly at the 
receiving end. 

The following codes were designed to match the statistics of the 


derived data: 
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Code 

Words 

A" 

L" 

A*"*" 

00 

Figure termination 

0 

0 

111 

N e 0 

N « 1 

M « 1 

0101 

1 

- 1 

- 1 

1101 

- 1 

2 

2 

1000 

2 

^2 

-2 

0111 

-2 

3 

3 ^ 

0110 

3 

-3 

-3 

1011 

-3 

k 

h 

0100 

positive 1 block 
extension (5 bits) 

positive 1 block 
extension (A bits) 

positive 1 block 
extension (5 bits) 

1100 

positive 2 blocks 
extension (10 bi ts) 

positive 2 blocks 
extension (8 bits) 

positive 2 blocks 
extension (10 bits) 

1010 

negative 1 block 
extension [h bits) 

negative 1 block 
extension (A bits) 

negative 1 block 
extension (i» bits) 

1001 

negative 2 blocks 
extension (8 bits) 

negative 2 blocks 
extension (8 bits) 

negative 2 blocks 
extension (8 bits) 


For tsT If N > 0 and ^ < H < 35 

36<N£l0G0 
If M < 0 k <|h1< 19 
20<!n|<275 


code length = 9 bits 
code length = 1^ bits 
code length = 8 bits 
code length = 12 bits 


Figure 3.2 Encoding A's by Extension Code 
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(a) Code - for Run Lengths 

This code was first developed by Huang, etc, [56] to encode the run 
lengths. The optimum Huffman code is impractical due to the fact that the 
run lengths may be any possible integer. Each code word consists of 
several fixed length blocks of two bits. Figure 3«3 shows an example of 
the Rj code; it consists of three blocks. The first bit in each block is 
the identification bit which is used to identify the starting and end of 
a code word. The second bit is the information bit. All of the identifi- 
cation bits of a code word have to be either all zero or all one. 


ID 


+ 

ID 

1 Infl + 

ID 

Linfl 


Figure 3*3 An Example of R^ Code with Three Blocks 

When a two-level picture is coded, the identification bit can also be used 
as the color code. For a multi-level picture, the expansion of one identi- 
fication bit into two or more bits is not recommended, since there is a 
better way to handle this problem as disucssed in chapter A. 

Because there is only one information bit per block, the total number 
of possible code words consisting of n blocks is 2^. The encoding algo- 
rithm is as follows: 

0) Determine the range of the run length L, 

If 2"-l^ L < 2"'^’ - 1 

Then L Is mapped into a code word with n blocks. 

(2) Determine the n information bits. 

The binary representation of Z coded in the n information bits is 
Z » L - 2” +1 . 



The major attraction of Rj code is the ease of implementation. A com- 
puter simulation with the Rj code will be presented in chapter 4. 

(b) Cj code - for PDQ or DDC (i.e. for A", A"" and &"') 

Through the predictive coding, PDQ or DDC, the input data (images) 
will be converted into sequences of index pairs: (a", &") or (&' p 

The range of these indices of the simulated pictures is from -260 to 10i»0 
During the first level of PDQ or DDC processing, the figure termination i 
represented by n+1 for a picture with n samples per scan line. 

The application of the same code to A" and &" or A" and may 

Simplify the implementation complexity of the encoder and decoder, but it 
also has a drawback, since no single code can be optimal to two source 
messages with different statistics. 

The encoding procedure for Cj code is as follows: 

Let A denote any index: (A", A" or A"'"') 

(1) The mapping Mj will map A's into N 

Mj : If Ajc 0 N = 2 X (-a) 

A>0 N = 2xA-l 

(2) The mapping will map N into Cj codes 

If 2 '^ -1 ^ N+1 < 2*^^^ -1, then the code word for N consists of n 
blocks. The first bit of each block is the Identification bit, 
followed by the information bit. 

( 3 ) The binary number Z coded in n. information bits of the code word is 

Z = N - 2 " + 2 

Figure 3.^ shows some code words In Cj code. The code length for 
figure termination is the longest. 
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(c) 0^ code - for A" only 

The statistics of A" for most of the simulated pictures shows that 
the probability of occurrence of the figure termination is very high. 
Therefore, it wili be more beneficial to construct a code mapping which 
assigns a shorter code word to represent the figure termination. Thus, 
we can apply the Oj code for A' and the Cj code for A""* or A'""* to achieve 
a better data compression ratio. 

The encoding algorithm for 0^ code is as follows: 

(1) The mapping maps A' into N 

: The figure termination is mapped to N = 0 

for A- > 0 N = 2 X A' 

A' < 0 N » 2 X (-A^) + 1 

(2) The mapping is the same as in code encoding procedure 
Figure 3,5 shows some code words of the 0, code where the figure 

termination is mapped into a code word with only two bits. 

For a picture with size greater than the normal operating size, which 
is determined by the practical situation and the capacity of the computer 
in use, it is better to divide the picture into several sections such that 
no lengthy code word will be used. And the segmentation of the pictue 
will also reduce the transmission errors which occur in the lengthy code 
words when the segmentation is not used. 

All of the codes developed in this chapter are simulated to determine 
their performance in chapter 


Oj Code - for A"* only 
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Figure 3«5 0i Code 





CHAPTER 

BACKGROUND SKIPPING AND ANALYSIS OF THE SIMULATED DATA 

Background skipping enables us to deal with the data v/hich has less 
probability of occurrence such that nx>re saving on the processing time 
and higher data compression can be obtained. The background class can be 
determined from the statistics of the picture. 

In this chapter, we will present some computer simulation results by 
applying those coding techniques and codes investigated in the previous 
chapters, to binary and multi-level pictures. The comparisons among these 
coding schemes and codes are also made. 

An analysis of the derived data from the computer simulation is 
described In section ^.3. Finally, a compressed data format is developed 
to pack the encoded indices (i.e. run lengths or A's) in an efficient 
manner that the data compression can be achieved with minimum errors. 

I>.1 Background Skipping Technique (BST) 

Definition of Background Class 

The major class of a picture with two or more classes is defined to 
be the class with the highest probability of occurrence. If the picture 
is too complicated and the statistics vary significantly from region to 
region, we can divide the picture into several sections, and define a 
major class for each section. Those classes which are not selected as 
the major class are defined to be the minor classes. 


We select the major class as the background class for a section of 
the picture or the entire picture depending on where the major class is 

f 

defined. 

Advantage of BST When Applied to PDQ and DDC 


The characteristics of PDQ and DDC is to transmit the picture with 
fewer digits by updating the the changes of the figures between the suc- 
cessive scan lines. However, if we randomly selected those figures repre 
sented by the major class, we will have to spend more time in the process 
Ing of encoding and decoding in addition to the detection of index pairs 
in the scan lines. When the multi-class picture is concerned, the waste 
in time and data compression will be more severe. 

With the background skipping, a class code for the background class 
Is transmitted at the beginning of data transmission. Then, the index 
pairs of the figures represented by the minor classes are transmitted. A 
the receiver, the storage space for the reconstructed picture is first 
filled with the background class. When the decoding is completed, the 
transmitted figures are formed in the appropriate locations. 

Table A.l shows the advantage of the application of the background 
skipping in the simulations, of PDQ where the processing time and the data 
compression are considerably improved. 

The statistics of the two-level pictures is presented in Table 4.2. 
Since the land is the major class, it is defined as the background class. 


4.2 Computer Simulation of PDQ and DDC 


There are two steps in the data compression routines of PDQ and DDC. 
The first level of compression converts the pictures into sequences of 


index pairs. A flow chart in Figure 4.1 shows the algorithm of the first 




Table 4.1 Advantage of Using Majority Class as Background Class In PDQ 



PDQ (Cj code) 

PDQ (C^ code) 

PICTURE 

Background Class 

Background Class 

5 ^ Majority Class 

=» Majority Class 

No Flood Reservoir 

R = 4.48611 

comp: 54.30300 sec 

decomp: 94.30300 sec 

R = 5.25203 
comp: 14.02 sec 
decomp: 53«204 sec 

4522 words 

1008 words 

861 words 

Flooded Reservol r 

R = 3.79222 
comp: 52.4 sec 
decomp: 76.415 sec 

R = 4.48004 
comp: 12.097 sec 
decomp: 44.255 sec 

3705 words 

977 wprds 

827 words 

Flooded Reservoir 

R = 0.80937 
comp: 92.841 sec 
decomp: 107.318 sec 

R = 1.00686 

comp: 52.538 
decomp: 79.928 sec 

6960 words 

8599 words 

6913 words 

No Flood River 

R « 7.48166 
comp: 46.829 sec 
decomp: 74.828 sec 

R » 12.9113 
comp: 6.525 sec 
decomp: 34.526 

3 O 6 O words 

j 

! 

409 words 

237 words 


I 















Icivel of compression for PDQ, The second ievel of compression converts 
the index pairs Into coded messages according to the mapping of the select 
ed code. 

When the first level of compression is completed, the index pairs 
of each scan line are arranged in the format as shown In Figure 


(A^,A^-'), 


N denotes the number of index pairs for this scan line 
Figure Arrangement of the Index Pairs 


During the first level of compression, we have used the following 
algorithm to find the indices: 

(1) Locate all the figures on the scan line and record the starting 
column and width for each figure. 

(2) Group the index pairs of the same class together such that only 
one class number is used :o identify each group. 

(3) For each class of the next scan line, find the figures as 
described in step (1) and (2) by looking to see if the figures 
are confined to their neighborhood, calculate A's. Then, find 
the new figures of each class, and locate the Index pairs. 

(i|) Repeat the step (3) for the rest of scan lines until the last 
one is reached. 

Recall that code words of the non-prefix codes (I.e. Cj code or 0^ 
code) consist of blocks of two bits. One of them is the identification 
bit which can be used as the class code for the two-level picture. There- 
fore, there is no need to pack a class code in front of the sequence of 
index pairs of the same class. When a multi-level picture is concerned. 





It Is not worthwhile to extend one identification bit into two or more 
bits. Instead, we can pack a class code in front of the group of index 
pairs of the same class. This will optimize the second level of compres- 
sion by saving a lot of identification bit which v^/ould occur in each block 
of the code words. 

described in Chapter 2, we have classified the mul ti spectral images 
Into (1) two-level pictures which include the river map, and reservoir map 
both In the regular and flooded seasons; (2) multi-level pictures v/hich 
include the land use map and snow cover map. 

These pictures were used in the computer simulations by various cod- 
ing techniques and codes which were discussed in Chapters 2 and 3. The 
performances of the simulated results were evaluated. 

For convenience, we divided the results into three parts - two-level 
and multi-level pictures plus the data statistics: 

(a) Simulation Results for Two-Level Pictures 

Table A. 3 shows the data compression ratios obtained through the ap- 
pll cations of Pvl.C, PDQ and DDC using the code, Cj code, and 0^ code, 
respectively. In general, the f looded-per iod pictures are more compli- 
cated than those of the regular season because there are more figures 
which contribute more new starts, figure terminations, and short runs. 

The advantage cf PDQ and DDC over RLC no longer holds in these pictures 
from the fact that the algorithms of PDQ and DDC are designed to take 
advantage of the small changes of the figure between successive scan 
lines. 

We made another simulation by implementing two options of the ex- 
tension code with PDQ and DDC. One of these Is to encode the index 


Table 4.3 Simulation Results for Two- Level Pictures 
where and Codes were Used 


Cod I ng 
Scheme 

Picture 

RtC (R,) 

POQ (C,) 

CDC (C,) 

1. No Flood Reservoid 
4522 words 
(1 word • 60 bits) 

R “ 5,4306 
Comp: 9.994 sec 
Deconp: 50.015 sec 
833 words 

R « 5.25203 
Comp: 14,02 sec 
Oecomp: 53.204 sec 
861 words 

R • 5.74587 
Comp: 14.655 sec 
Deconp: 52.174 sec 
787 words 

2. Flooded Reservoir 
3705 words 

R « 4.28979 
Comp: 8.445 sec 
Oecomp; 40.439 sec 
864 words 

R « 4.48004 
Comp: 12,097 sec 
Oecomp: 44.255 sec 
627 words 

R == 4.77448 
Comp: 11.735 sec 
Oecomp: 41.776 sec 
776 words 

3. Flooded River 
6960 words 

R - 2.15597 
Comp; 22.206 sec 
Dcccmp; 78.008 sec 
3228 words 

R = 1.00636 
i Comp; 52.533 sec 
Oecomp: 79.928 sec 
6913 words 

R = 1.071^6 
Comp: 50.S93 sec 
Oecomp: 78.103 sec 
6381 words 

4, No Flood River 
3060 words 

R » 9.5625 
Comp: 5 •632 sec 
Oecomp: 33.087 sec 
320 words 

R = 12.9H3 
Comp: 6.526 sec 
Oecomp: 34.526 sec 
237 words 

R «= 14.641] 

1 Comp; 6.467 sec 
Oecomp; 33.119 sec 
2C3 words 


Comp = CPU Time Used for Compression 
Decomp = CPU Time Used for Decompression 
R = Data Compression Ratio 


PAGE IS 
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pairs with the same version of the extension code. The other Is to select 
two different versions of the extension code to encode h' and h" (or h'") 
respectively. Table A. gives a comparison of the performances between 
these two options. It Is obvious that the second option Is definitely 
better. 

We also simulated the two-level pictures with PDQ and DDC, where Oj 
code was used to encode A"*, and Cj code was selected for encoding of A'”' 
or A'*'"'. As shown In Table If. 5, we see that the data compression ratios 
achieved are better than that shown In Table If, 3 where the Cj code was 
used to encode all the Indices (A", A'"’, and A'"'*). 

Table If. 6 shows a list of entropies of run lengths and A's In addi- 

tion to the optimal data compression ratios we can achieve. This table 
Is prepared to be used as a reference to evaluate the codes we have dis- 
cussed. The optimal code known to us is the Huffman code, but It Is not 

practical in the coding operations. We found that the data statistics 
for the same type of p^lcture have the same characteristics. Thus, we can 
predetermine the code for each type of picture. From the simulations, we 
are convinced that the codes v/e have developed can be used to operate In 
an efficient manner without the knowledge of the statistics of the In- 
dividual picture. For example, RLC operates much better than other tech- 
niques for the flooded river maps, while DDC and PDQ are the best for the 
three other types of pictures. 

(b) Simulation Results for Multi-Level Pictures 

Among the multi-level pictures we simulated, the snow cover map Is 
considerably more complicated than the land use map even though there are 
more classes in the land use map. We also noted that the data compression 
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Table ^«,5 Simulation Results where 0. Code 
■ Code is Used for L" & ' L'" 


PICTURE 

PDQ, 

1 No Flood Reservoir 

670 words 

^522 words 

R = 6.74925 

2 Flooded Reservoir 

797 words 

3705 words 

R = 4.64852 

3 Flooded Reservoir 

6523 words 

6960 words 

R = I.O67O6 

4 No Flood River 

229 words 


3060 words 


R « I3.362A 


527 words 
R = 8. 5806 A 

71^ words 
R = 5.18553 


5338 words 
R = 1.36379 
201 words 
R = 15.2238 










Table k,S Entropy Resulted from Various Coding Schemes 


PICTURE 


J No Flood Reservoir 


259290 bits 


2 Flooded Reservoir 


E(L)-5.6<(37 bits 
Entropy ■ 3852l| bits 
R = 6.7306 


E(L)= 5 . 8 ^i 84 bits 
Entropy = 38302 bits 
R ■= 5.^897 


210270 bits 


E(i')-3.2I2I bits 
E(i")=3.8829 bits 
Entropy = 23665 bits 
R = 8.7^06 


E(A')^I|.877I bits 
E(A")=i|.lH55 bits 
Entropy =• 35*11^ bits 
R: »= 5.9375 


E(6')-3.2I2I bits 
E(A'")»2.7237 bits 
Entrooy = 2ii839 bits 
R “ lb.it383 


E(a')=I(. 8771 bits 
E(A'")=3.1'ii3 bits 
Entropy ™ 3i*»73 bits 
R = 6.6609 


3 Flooded River 


li056i*0 bits 


E(L) - i|.2678 bits 
Entropy •= 1337^5 bits 
R = 3.0329 


£(a')= 6.3333 bits 
E(A")='t.39''I bits 
[Entropy => 2915i|8 bits 

R = 1.3913 


E(a')= 6.3333 bits 
E(a'")- 2.5929 bits 
Entropy = 2 h 2537 bits 
R = 1.6721 


No Flood River 


171600 bits 


E(L)=7.3967 bits 
Entropy =13255 bits 


E(a')= 3.6020 bits 
E(a")= 3.2269 bits 
Entropy = 5969 bits 
R = 28 . 7^85 


£(a')= 3.6020 bits 
E(A'")=2.9276 bits 
Entrooy = 5707 bi ts 
R - 30.0683 


E(L) = Entropy of run length E(a^) = Entropy of L' 
R = Optimal data compression ratio 
Entropy = Entropy of the entire picture 


ORIGlNAi; PAGE IS 
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ratios achieved by PDQ and DDC are not as good as that achieved by RLC. 
Furthermore, it shows that the complexity of the figures of the picture 
is a major factor in the data compression ratios even though the number of 
classes (or grey levels) in a picture is somewhat important. 

As shown in Table ^.7» the code and the code were selected for 
the simulation of multi-level pictures. We have come to a conclusion 
that RLC is more efficient than PDQ or DDC as far as the multi-level pic- 
tures are concerned. Especially, when there are lots of figures v/i th high 
complexity in the picture, any error free coding technique such as RLC, 

PDQ or DDC will not serve as a good technique for data compression. 

Some coding schemes with distortion like the one suggested by Ziv 
I 57 ] niay be the al ternative which provides the constant bit rate at the 
output of the encoder. This scheme considers blocks of data symbols of a 
predetermined length, and assigns a string of binary digits, that has a 
predetermined length, to each block. The length of the blocks depends on 
the output bit rate desired. This technique is not error free because a 
single code word was assigned to a number of different messages. 

(c) Data Statistics 

Figure i».3 through Figure k.S are the measurements of the various 
data statistics for binary pictures. The curves represent the envelopes 
of the relative probability of occurrence. Numbers shown on the abscissa 
represent the extent of the data. The vertical axis represents the 
relative probability. Thus, the cases with zero probability of occurrence 
do not imply that they did not occur in the picture. Instead, they are 
just less likely to occur with a probability close to zero. 

Comparing the probability distribution of L'" against that of A''"*, 
we find that there is a dominant spike located at the center of the curve 



) 
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Table 4.7 Simulation Results for Multi-Level Pictures 


Pi^ture^''^ 

RLC (R^) 

PDQ (C^) 

DDC (cp 

Snow Cover Map 
3134 words' 

(3 Levels) 

R = 1.85443 
Comp; 18.273 sec 
Decomp: 3^.066 sec 
T690 vvords 

R » 1.04142 
Comp: 28.214 sec 
Decomp: 41.42 sec 
3009 words 

R = 1.27205 
Comp; 27.473 sec 
Decomp; 39.572 sec 
2464 v;ords 

Land Use Map 
1548 words ' 

(5 Leve 1 s ) 

R = 3.23849 
Comp: 15.518 sec 
Decomp: 17.653 sec 
478 words 

R = 1.4452 
Comp; 31.567 sec 
Decomp: 23.823 sec 
1071 w'ords 

R = 1.76197 
Comp: 30.321 sec 
Decomp: 20.483 sec 
879 words 

1 — : 

Comp = Compression Time Decomp = Decompression Time 


R = Data Compression Ratio 







100 200 300 -100 0 100 700 
Run length 


Figure h,k Envelope of the Probabil I ty Distribution 
of Indices of Picture 2 
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of A'*'''', however, there Is no such spike presented In the curve of L" . 

From the entropy point of view, this Is rather Important, The entropy for 
U'" win be less than that of h" based on the characteristics of the 
entropy function. Furthermore, most of the derived data as shown In 
these figures has the same characteristics in the probability distribution 
which will make the design of efficient source codes a lot easier. Note 
that. In the probability distribution of A^, there Is always an Impulse 
located at the end of the abscissa. This represents the probability of 
the figure terminations of the entire picture. 

The figures also imply that It will be more efficient to apply dlf- 
ferent codes for different Indices if the system Is allowed to Implement 
several options of codes In the encoder. 

4.3 A Mathematical Model for the Derived Dat a 
The probability density functions of run lengths and Indices as shown 
irj section 4.2, are exponentially distributed with parameter X. The curves 
for the probability density function of A'*" and A'""' are roughly symmetric 
with respect to a dominant spike which Is located at the origin of the 
horizontal axis. 

Let the probability density function of A be 

P(X) = X e"^^ (‘‘•D 

where X Is the parameter, and the Integer X Is the absolute value 
of A. 

The probability of X to be between I + 1 and I Is 
P, » prob { I <X < 1+1 } ® 2 


P(X) dX 



Define a = e ^ N I 1.1 
Equation (4.2) can be written as 

P. = 2(l-a) a', for i = 0,1,2,... (4.3) 

The entropy per source symbol. A, is 

H = -E P. log P. (4.4) 

i 

From (4.4) 

H = - { 2(l-a) log22(l-a) + 2(l-a) a log22(l-a) a + ...} 

« - { log 2 [ 2 (l~a)]^^' log^[2 (l-a)a]^^' ... ) 

= -{ log2[2(l-a)]2<’-3> [2(l-a)a]^^’"^^^ ... } 

_ _ ^ ^Qg [2 (l -a) ] ^ ■*" 3 ^+...) ^2 (l~a)a+4 (l"a)a +...j 

(4.5) 

Since 2 (l-a) (1+3+a^+aV ...) = (1-a) = 2 

I "3 

and 2(l-a)a+4(l-a) a^+6(l-a) a^ + ... 

= 2(l-a)a[l+2a + 3a^ + ...] 

= 2(l-a)a — 

(1-a)^ la 

Thus, 

2a 

H = - log2[2(l -a)3^ a^^] 

» - 2 [log 2 2 + log^d-a)] - log 2 a 

« -2 + 2 log„ T~- + T~ log., -r (4.6) 




63 


To find the upperbound for the entropy function H, 

let us consider a 

positive constant X. Xj^O 


Since An (l+X) ^ X 


1 .e. e ^ (j+x)_;^ 1 


or Xe 1-e ^ 

(4.7) 

From (4.7) 1 

1 - e"'^ 

(4.8) 

Again, let a = e ^ then An a ^ = X 



Equation (4.8) can be written as 


in (-) < 2 

I -a - — 


a — 


2a 


2 

or (-) < e 


Multiplying (^.10) by (rTr) 


2a 

1-a 


(i-) (i^) < (-nj) 


1-a' 
2 


Taking logarithn of base 2 on both sides of eq. (4.11) 
we have 

2a\ /1\ i__ 1 


-2 1c 92T^<2 log,(^) 

From eq. (4.12) and (4,6) 
we conclude that 

H < 2( log-(T?r)- 0 


(4.9) 

(4.10) 


(4.11) 


(4.12) 


^.13) 


Compressed Data Format and Block Packing 
As the redundancy of the images was removed or reduced by the redun- 
dancy remover and encoded with the adequate variable length codes, the bit 

I 

string has to be packed in. an efficient format such that the transmitted 
data can be decoded correctly at the receiver. Furthermore, the compressed 
data format protects the data from the serious error propagation and con- 
fines the effect of the errors to the minimum extent such that the re- 
production of the image can still be obtained with minimum error. In fact, 
the derived data from the data compression is usually very sensitive to 
the errors in the sense that if some of the derived data was corrupted, 
the original picture can not be faithfully reproduced. In other words, 
one single error in the derived data of a scan line may corrupt the rest 
of the scan lire which will displace the figures in the picture. 

We present a compressed data format for the derived data of PDQ or 
DDC in Figure k,7> 


k- 



words for scan line 




Ia 





^2 

(A^A''"),(A" A"-*),... 

k- 



W- words for scan line i 
2 




1 



(A", A"") ... 


W. = word count 

. I 

Cj = class code for class i 
N. = number of index pairs 

I 


Figure 4.7 Compressed Data Format for PD(1 or DDC 





The number of bits for W. depends on the size of the image and the 
complexity of the figures. The length of Wj has to remain the same through 
all the scan lines. We can use the natural code as the class code since it 
will be more convenient to use a fixed length code to encode the class 
number. The chance of errors will be reduceji. The number of the index 
pairs, N., is used to ensure the number of transmitted index pairs which 
is very helpful in the decoding process, * 

To initiate the new starting figures in a scan line, we pack the 
index pairs in a format so that no special code word for the initiation 
of new starting figures is necessary. 

Let us consider Figure k.B as an example. 


Figure *1,8 An Example of Three Figures in a Picture 


We note that there is a new starting figure in scan line 2. Figure 
1 terminated on scan line 3. When the indices of the figures were found, 
they are arranged by the sequential order that index pairs for the old 
figures were packed first according to the sequence that occurred in the 
scan line, followed by the index pairs of the new figures. 

Thus the index pairs for the four scan lines are arranged as follows 
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K (a; ,Ap(A£^Aj") 

2. (AJ, AJ')(AJ, A^")(A^, AJ") 

3. (A^, a^HaJ, a^-)(a^, a^-) 

k. (Aj, e^'nUy A^") 

Note that In the scan line 2, A^ denotes the starting column number, 

while A^'* denotes the width of figure 3. In scan line 3, A^ represents 
the termination code (i.e., n+1 , if there are n samples per scan line), 
and Aj’" represents the width of the merging figure 1. 

The number of index pairs varies from line to line. The indices are 
encoded by the variable length code and packed into the compressed data 
format. Thus, the length of the compressed data format for each scan line 
also varies from line to line. 

In order to transmit the data string packed in the compressed data 
format at a constant bit rate, we have to divide the data string into 
fixed length blocks. Each block consists of fixed number of words. And 
the number of bits in a word depends on the computer. Through the help 
of the word count packed in front of the sequence of Index pairs for each 
scan line, the start and end of the scan line still can be recognized at 
the receiving end. 

Once the complete information of a scan line was received at the re- 
ceiver, the decoding process can be done as follows: 

Step (1) Read the number from word count, W. . 

Step (2) Decode the class code and the number of figures. 

Step (3) Decode the index pairs. 

Step (i») Update the column number and width for old figures. 


Step (5) Locate the new starting column number and width for the new 


I 

l 

t 

I 

\ 

I 


figures occurring in the same scan line. 

Step (6) Repeat Step (2) to (5) if a multi-class picture is concerned. 
Step (7) Reconstruct the scan line with appropriate symbols. 

Step (8) Repeat Step (1) to (7) until the last scan line is reached. 
Thus, an error free reconstructed picture can be obtained provided that 
the channel noise was not introduced in the data transmission. 




CHAPTER 5 


DATA COMPRESSION ON SATELLITE IMAGERY 

The image taken by the remote sensing satellite Is usually a two- 
dimensional array of sampled data. Each element of the array Is a pic- 
ture element which is quantized intom-bits of data. Thus, each picture 
element assumes one of "T grey levels. To transmit a picture of size 
Hxn, we have to transfer mx*.xn bits per channel. For most recent multi- 
spectral satellite data, 7 bits per picture element is used. Those 
pllcturcs with better signal to noise performance were compressed into 6 
blits per picture element by a nonlinear scale. For example, pictures in 
channel 1 to 3 of the four-channel LANDSAT pictures were compressed be- 
fore they are transmitted [58], When the transmitted image is received 
at the ground station, it is requantized into 7 bits of data and one 
zero per picture element through an algorithmic scale. 

With multisosctral satellite images, the total number of binary 
digits needed to be transmitted is great. The data will be transmitted 
at 120 mega bits per second, which is the rate designed for new LANDSAT- 
0 THERMATIC MAPPER. Only a few ground stations are presently able to 
handle such a high transmission rate. In order to make those ground 
stations with lower receiving capacity accessible to the system, some 
effi cient data compression schemes are expected to compress the data 
such that the same amount of information can be transferred to the 
ground at a lower transmission rate in the same period of time used by 


the high transmission ratCo This will cjanble the satellite to transmit 
the data at several different rates simultaneously, A satisfactory 
representai ton of the digital image (eog, satellite images) requires 
more binary bits than the entropy of the image. Fortunately, the re- 
dundancy presented in the imagery data can often be reduced through 
various coding schemes. This chapter deals with those promising coding 
techniques which are best suited to our goal based on the practical 
point of view and low-cost principle. We also assume a general appli- 
cation of the received images. Therefore, those techniques which deal 
with the pictures after classification is done are not considered, 

5,1 Definition of Our Goal 

As the information extracted from remotely sensed satellite images 
is shown to be very valuable and important for the Investigation of 
man's environments and estimation of the earth's natural resources, 
many nations in the v/orld have become more interested in participating 
in the program of a near real time data acquisition system, A high data 
transmission rate would be necessary to transfer ..the enormous data bulk 
from the satelliteo The LANDSAT-D Thermatic Mapper Satellite which is 
planned to be launched in I98O will operate at the rate of 120 mega bits 
per second. It can be expected that the cost of a high quality ground 
station able to receive such a high bit rate would be rather expensive. 
Those nations which cannot afford the cost will be unable to access the 
system directly. 

Based on the assumption that the application of the received sat- 
ellite data is for a general purpose, our investigation is destined to 
provide three data transmission rates di rectly from the satellite such 
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that the ground stations with receiving capability of 120, 30 and 20 
mega bits per second are all able to receive the data directly from the 
satellite. In other words, data compression ratios of k and 6 are de- 
sired to reduce the rate from 120 mega bits to 30 or 20 megabits, 
respectively. 

A certain amount of distortion is introduced during the picture 
coding in addition to the channel noise in the picture transmission 
system. Here, we are mainly concerned with the performance of the 
various coding schemes for the digitized data. Therefore, we defined a 
fidelity criterion which Is used as a measure of the reconstructed 
picture quality when compared to the original picture excluding the 
channel noise. 

Suppose that the original picture consists of a n x m array of 
picture elements X.., where i = 1,2,. ..,n, j = l,2,...,m. Each picture 
element X.^ Is K binary bit word corresponding to one of the 2' possible 
grey levels. The encoder compresses the original picture of nxmxK bits 
to a fewer number of bits. The decoder interprets the received coded 
messages and reconstructs the picture consisting of nxm array of 
picture elements Y.j, where i = 1,2 j = l,2,...,m. 

The root mean square error between the original and reconstructed 
picture is 


rms 


, n m - JL 

—L_ [ I s (X - Y..)'^]2 

n*m ' ij ij 


(5.1) 


We will use this fidelity criterion to evaluate the performance of 
picture compression techniques discussed in this chapter. However, the 
human vision system has a characteristic that two reconstructed 
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pictures having the same amount of rms error may appear to be quite 
different to human observerso Indeed, the human vision system is more 
sensitive to errors in dark areas of the picture than errors in light 
areas. Therefore, we would not suggest to employ human observers in 
judging the rms errors in various reconstructed pictures. 


5.2 Logarithmic Quantizing 

Recall that a satisfactory picture usually requires 7 or more bits 
per picture elements. Since the amount of the image data needed to be 

I 

transferred is enormous, it will be more economical to compress the 
image by utilizing a nonlinear quantization scale and decoding the re- 
ceived message to reconstruct a picture with 7 bits per picture element. 

Since the distribution of the digitized sample data is nonuniform, 
and its distribution varies from one spectrum band to another. In order 
to compress the data from 7 bits into 6 bits per picture element, it is 
reasonable to adopt a nonuniform quantizer that increases the size of 
intervals which are unlikely to be occupied, and reducing the size of 
those whose usage is highly probable. This will result in a least rms 
error. 


Indeed, this first level of data compression has been used on 
LANDSAT satellite images. The decompression of 6-bit picture into 7“bit 
data was done at a NASA ground station. Now we will demonstrate how 
the logarithmic quantizing scale is developed. 

Let the horizontal axis or X-axis represent the nonuniform 
intervals, and the vertical axis or Y-axis represent the uniform 
intervals with v/idth one unit. On the X-axis, let the 1th interval be 
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of width 6. centered on the value X., Therefore, any sampl 


e amp) i tude 


lying In the range, 


X. - ■J- 6. < X < X. + i 5. 
I 2 I — • 2 I 


(5.1) 


will be represented by the amplitude X.. Thus, the instantaneous 


squared error is (X - X.) , Let the probability density of the ampli- 
tude distribution of the sampled data be P(X), Then the squared error 
due to signals falling within this interval is 


X. + 1 6. 

2 i 2 I 

^i = 1 


(X - X.) P(X) dx 


(5.2) 


1 8 , 


I 


P(X.) / 


y2 = ^ P(X.) jS 


The probability that the sampled data falls within the i th interval is 


X, 


P. A / 

I — Y - J. A 
^1 2 ^ 


1 


P(X) dx = P(X.) 6. 


(5.3) 


From (5.2) and (5.3) 


2 1 2 


(5. A) 


Let the curve on the XY plane v/hich will map the uniform scale into non- 
uniiform scale and vice versa, be defined as the companding curve, (it 
wi(l be named as the logarithmic curve due to the logarithmic quantiz- 
ing). The slope of a chord joining neighboring points on the companding 
curve is approximately the slope of a tangent to the companding curve 
at the mid-point of the interval. 


(•— ) 

' dy ^ 


(5.5) 


Equation (5.4) is the noise power due to nonun I form quantizing, 
stituting (5.5) Into (4,5) and summing over all the Intervals, 


E^- = 1 Z P 6 j = Z P. (-^)^ 

12 .1 . 12 . I dy 


(5.6) 


With many levels In the scale, the summation can be approximated by the 


integral 


/ P(X) (-^)^ dX 


(5.7) 


The signal power In terms of the probability distribution P(X) is 


= /P(X) dX 


Thus, the signal to noise ratio is 


^ = /P(X) X^ dX 

We see that, if for certain constant K . 


(5.8) 


(5.9) 


then S /E is Independent of P(X), or 


( 5 . 10 ) 


The companding rule which v/ould give this is found by integrating £q, 


(5.9) 


Y = K log X for x > 0 


(5. 11 ) 


7 ^* 


For a practical reason, Eq, (5.11) is usually modified into = K ^ log 
(l +pX) which displace the origin of the XY plane to the intercept on 
the X axis. 

Indeed, the logarithm quantizing was used widely in the speech 
processing especially in the telephone. For further development in 
this subject refer to reference [ 59 ] . 

Generally speaking, the advantage of the nonuniform quantizer over 
the uniform quantizer is that it has more quantization levels in the 
region which is more likely to occur. This, in turn v/ill reduce the 
mean square error. 

5,3 DPCM 

DPCM systems are based on an invention by Cutler [36 ]. In his 
invention, the quantized difference between successive sample values 
were transmitted rather than the sample values themselves. Oliver [60 ], 
Kretzmer [28], and Harrison [29 ] had proposed that the application of 
the linear predictor in the feedback communication system wi 1 1 reduce 
the redundancy in the data, and lower the required power in highly 
periodic signals, or equivalently, achieve some level of data compres- 
sion, 

A simple block diagram of the DPCM system is shown in Fig, 5.1. 

X(t) is the si gnal sensed by the sensor implemented on board the satel- 
lite, The sample selector provide the function of selecting the picture 
elements to be processed. * This sitUcition occurs v/hen it is necessary 
to skip some of the picture elements in order to achieve the required 
data compression ratio. 



CHANNEL 



Figure 5.1 Block Diagram of a Modified DPCM System With Sample Selector 
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Most pictures contain two types of redundancy in picture elements. 
One of them is the redundancy due to the high correlation between pic- 
ture elements. It can be removed by the differential pulse code modu- 
lation system. T„ Kailath [6l ] said in his paper that every signal 
can be conceived as having a predictable part which contains no in- 
formation and a nonpredictable part which contains all the information. 
DPCM di scards the predictable part and encodes only the nonpredictable 
part and reconstructs the predictable part at the receiver. 

The second type of redundancy is due to the nonuniform probability 
density of the nonpredictable part of the signal. We can remove this 
type of redundancy by entropy coding using a variable length code. 
However, in order to do the entropy coding, the probability density 
function of the data has to be found which may require a buffer to store 
the signal, thus, some delay would be necessary. 

5.3.1 Linear Predictor 

2 

Let a stationary signal X(t) with zero mean and variance a be 
sampled at the times t^ t 2 ,...,t^... and let the sample values be 
Xjt X 2 ,...X^i.o. respectively. 

A linear estimator of the next sample value X^ based on the pre- 
vious n sample values X,, X„,...X , is defined to be 

12 n 

L = C,X, + C X + ... + C X . 

G I I 2 2 n n 

For simplicity, we assume that C's and X's are real numbers and integers 

A 

respectively, A linear predictor provides the estimate X^, and trans- 
mits the difference (or error) 




Xq is defined to be the best estimate of fop which the expected 
value of the squared error is minimized. 

The optimum choices of the coefficients C's which satisfy the min- 
imum mean square error criterion can be found by taking the partial 
derivatives of E[(Xq - X^) ] with respect to each one of the C's and 
setting the equation to zero, we are ready to have 

E[(Xq - (CjXj + C 2 X 2 + ... + C^X^) X.] = 0 (5.12) 

or 

E[(Xq - Xq) X.] = 0 i = 1,2, ...n (5.13) 

Let R. . = E[X.X.] be the covariance of X. and X. then the condition 
U ' J I J 

for the optimum linear estimate is 


*^01 “ ‘'I'^li S'^Zi *'* * ‘^nVi 


I ” i,2,...ne 


The sequence {e.} is less correlated and has smaller variance than 


the sequence' (X.) from the fact that 


or 




o^- = R-. - (C.R-, + ... C R- ) 
e 00 1 Ol n On 


(5.1^*) 


(5.15) 


If the sequence (X. } is a first order Markoff sequence then only 
one previous sample is needed to be used in finding the best estimate 


V 


5 . 3.2 DPCM with Natural Code 

The difference between the actual sampled value and the linear 
estimate is quantized into one of 2^ levels (N bits) either uniformly 
or nonuni formly. The qunatizer represents each value of e. by the 


N 

nearest one of 2 quantizing levels. Then, a natural code is used to 
encode each quantizing level with N binary bits. A buffer may be neces- 
sary to store the coded bit string, then transmit at a constant rate 
over a channel. The received quantizing levels are used to reconstruct 
the picture. The difference between e. and the quantizing level used 
to represent it is called the quantizing error, q.. Thus, the messages 
being transmitted are the sequence {e. + q.}. 

Our goal is to achieve the data compression ratios of four and six 
under the constraint of minimum root mean square error. In other words, 
compressed pictures with average 1.5 bits and 1 bit per picture elements 
are desired. The natural code provides a convenient encoder which maps 
each quantizing level into a fixed length code word (N bits per code 
word if a N-bit quantizer is used). Suppose that a picture processed by 
the regular DPCK is transmitted with 6 bits per picture element. A 
modified DPCM as shown in Fig. 5.1 is designed to provide the desired 
compression ratios. With the natural code, the data compression ratio 
is fixed once the n-bit quantizer is determined no matter v/hat quantiz- 
ing scale is applied. 

We have simulated the SKYLAB IV pictures on the computer by the 
schemes discussed later in this section. Figs. 5.2 to 5.6 are the 
original SKYLAB IV pictures which were taken in January of 197^. These 
pictures provide information about the snow cover on the ground. For 
comparison, Fig. 5.2 will be used in computer simulation for various 
schemes. 

The histograms of these pictures are shown in Figs. 5.7 to 5.11. 
Figs. 5.12 to 5.16 show the histograms of the differences between the 
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Flgure5ol3 Histogram of the Differences of the 
Neighboring Pels for Simulated LAND 
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Figures. 15 Histogram of the Differences 
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neighboring picture elements (I.e,, X. - Some statistics of the 

SKYLAB IV pictures were presented in Table 5.1 through 5.2. 

From the histograms of X. - we can see that the probabMIty 

of the difference between neighboring picture elements v/hich ranged 
from -10 to 10 Is rather high. To minimize the rms error due to quanti- 
zation, an optimum nonuniform quantizer with unequally spaced levels 
will be the best candidate. Some sophisticated methods suggested by 
Max [37 ] are very attractive. However, they are still too complicated 
In practical operations. Indeed, our simulations showed that DPCM with 
an optimum uniform quantizer did provide reasonably good results. 

Although they are slightly Inferior to the DPCM system with anonuni- 
fbrm quantizer. In order to orovide almost real time transmission, and 
to eliminate the time delay due to calculation of the data statistics, 
and nonuniform quantizing, a DPCM v^ith optimum uniform quantizing will 
be a better system. 

The optimum uniform quantizer v;lll map the data Into one of a finite 

set {V,,...Vu) of integers: 

I M 

In other words 


Uj< X. ^1 " ''j 


(5.16) 


The set {IJ.} Is called the transition levels, and the set {V,} is 

J 

the quantization level. We defined -Uj = r where r Is an integer. 

The quantization value Vj is assigned to 

Vj»Uj+K‘r (5,17) 

where K Is a real number with magnitude less than 1. 

Then 


- Vj « Uj^^ +K*r- (Uj +Kt) » r 


( 5 . 18 ) 


Table 5.1 Statistics of the Difference of 
Neighboring Pixels of SKYLAB IV Images 


Channel 


Mean 


Standard *j 
Deviation i 


12.182 


11.782 


8.180 


6.08 


1.03 



Table 5.2 Statistics of Some SKYLAB !v Images 


Mean 

Standard 

Deviation 

31 

13.01 

17 

9.28 

9 

6.05 

9.1 

1*.01 

20.07 

1.5 










^ In the modified DPCM, a sample selector Is used to choose the 

picture elements to be processed by the OPCM system when skipping some 
picture elements is desired. In general, most of the multispectral 
satellite images are over sampled by a factor of 1.5 to 1, This will 
enable us to skip some of the picture elements without causing too much 
rms erro'". 

We have developed four compression schemes which utilized the mod- 
ified DPCM system with natural' code. Those skipped picture elements were 
reconstructed by linear interpolation method, 

DN 1: Every other picture elements were processed by the modified 

DPCM system with a 3-bit quantizer. Theiskipping factor is 
2/1 and the data compression ratio achieved is A, 

DN 2: A one-bit quantizer Is used with the DPCM system. The trans- 

mitted signal only initiates the sign of the error sequence 
{Cj}. In fact, this is a delta modulation system. The 
skipping factor is l/l and compression ratio, R, Is 6. The 
rms error of the reconstructed picture depends on the choice 
of Kr, as shown in Eq. (5.17). If the successive picture 
elements do not change rapidly and the sampling rate Is high 
enough, this scheme may operate reasonably well. In general 
i t does not seem to be adequate to apply this scheme to any 
I picture with which v^e are dealing. 

DN 3: Apply every other picture elements to the modified DPCM 

sp tern with a two-bit quantizer. Thus, the skipping factor 
Is 2/1, and R = 6. In this scheme more quantization levels 


are provided to reduce the rms quantization error. 
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DN A modified DPCM with a ij-bit quantizer is used. The picture 
is divided into biocksi each block is a 2x2 array of picture 
elements. Only the top left picture element of each block 
is processed. The skipping factor is A/1. A data compres- 
sion ratio of 6 is achieved. 

For comparison, we present the performance of these four schemes 
with uniform and nonuniform quantizer in Table 5.3 and 5. A respectively. 
The results were obtained by simulating compression schemes on the 
SKYLAB IV pictures. Notice that skipping more picture elements during 
the processing will enable us to use a quantizer with more quantization 
levels, but it also has a drawback that some more error may occur during 
the interpolation to obtain the skipped picture elements. In general, 
the extent of skipping factor really depends on the sampling rate of the 
picture. The reconstructed SKYLAB IV pictures using a uniform quantizer 
were presented in Figures 5.17 through 5.20. 

5.3.3 DPGM With Variable Length Code 

To make the best use of a variable length code, we have to compute 

the histogram or the probability density function of the error sequence 

{e.}. The variable length code is optimized in a way that the error 

message, ej, v/ith higher probabi 1 ity of occurrence is mapped to shorter 

code words, while rarely occurred error messages are assigned to longer 

code words. Suppose that a N-bit quantizer is used in the DPCM system. 

N 

The possible range of the error sequence would be divided into 2 trans- 

N 

It ion levels with non uni form space. Meanwhile we also have 2 quanti- 
zation levels. code which was discussed before in previous chapter 

will be used as a variable length code in our computer simulation. 






















Figure 5»I8 Reconstructed Picture From DN 2 Compression 
Scheme 






Figure 5.19 Recc^ ' ructed Picture From DN 3 Compression 
Scheiiiu 





Figure 5.20 Reconstructed Picture From ON Compression 
Scheme 
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Among all the code words of code, there are only two code words 
of length two bits, which are the shortest. Thus, the performance of 
2-bit and l-bit quantizer with code will definitely be inferior to 

the natural code. Also, the length of the intervals v/hich are mapped 
into these two short code words v/i 1 1 be an Important factor in the over- 
all data compression ratio. That is to say, assigning two long trans- 
ition levels to two shortest code words will provide better data com- 
pression ratio. However, it also will cause a higher quantization 
error and rms error in the reconstructed picture. 

The probability density functions for the error sequence {cj } 
shown in Figs, 5,12 to 5.16 are somewhat symmetric. They may be ap- 
proximated by the exponential density function; 

p(e) = -1- exp( - — |e!) (5.19) 

2/r "" 

2 

where a is the variance of the error sequence. 

Since the probability density function is different for each picture, 
the best we can do is to choose some representative density function 
and match the quantizer to it. 

One simple way to obtain the proper quantization and transition 
levels for minimizing the rms quantizing noise is to form a function, 
f(a)» such that when e takes some value in the range of V to -V, f 
assumes a proper value* Thus, a table is formed. We can divide the 
range of -V to V into 2^ transition levels through the help of this 
table. The quantization value, V. , between U. and U.^l in the transition 
axis can be assigned as 

V, = i (u, * u,.,) 


( 5 . 20 ) 


]0k 


A paper by Smith [ 62 ] showed that when the probability density 
function of the signal to be quantized is that of Eq. (5ol9), the 
function f(e) is given by 

f(e) = - ” ^-nfi ~ |-(i-exp (-m))} 0<_e (5.21) 

f(-e) = f(e) (5.2:5 

where m - /Z M/'ia 

This scheme is rather simpie compared to those more sophisticated 
method such as Max's optimum quantizer. A table of the Smith nonuniform 
quantization is shown in Appendix. 

We have designed the following schemes ip which the modified DPCM 
system utilizing a nonuniform quantizer and encoder. 

DB 1 : A 3"bit quantizer is used with the modified DPCM system. 

The picture is first divided into blocks of 2x2 arrays of 
picture elements. Only the top left pels of each block are 
processed. The skipping factor is A/1, The system is ex- 
pected to achieve R=6 or better. 

DB 2: This scheme is the same as DB 1 except a 2-bit quantizer is 

used in DB 2. The skipping factor is A/1. We expect this 
scheme to achieve slightly higher data compression than DB 1. 

DB 3: Apply the modified DPCM system with a 3-bit quantizer to 

every other picture elements of each scan line. The skipping 
factor is 2/1. This scheme is expected to achieve R=A or 
better. 

DB A: A modified DPCM system with a A-bit quantizer is applied to 

every other picture elements. The skipping factor is 2/1. 

It is expected to achieve R-A or better, and the quantizer 
noise would be reduced compared to that of DB3. 


The performance of these four schemes is shown in Table 5.5« The 
reconstructed SKYLAB IV pictures v/ere presented In Figs. 5.21 through 
From the simulation results, we found that the variable length 
code (C^ code) did provide better reconstructed pictures by using more 
quantization levels and a nonuniform quantizer. However, It also has 
a drawback that the computation to accumulate the data statistics for 
the error sequence {Bj } and the nonuni form quantization may cause some 
time delay. 

It is really hard to draw a conclusion that the application of the 
variable length code will be better than the natural code, v/hlle imple- 
menting these compression techniques in the satellite to provide multi- 
rate data transmission. The most important factors in the decision mak- 
ing are the extent of time delays (due to accumulation oF data statistic 
for the error sequence) and the buffer size required to store tne data 
during the processing. Our simulations on the SKYLAB IV picture showed 
that for a picture of size 256x256, it will take 200 to 2^0 epu seconds 
to obtain the' error sequence statistics. In general, most of the 
SKYLAB IV pictures are of size 119^x22l»0 or larger, and it will require 
more than two hours of epu time to find the data statistics for the 
error sequence v/hich is not adequate for a real time data transmission 
sys tern. 

5.4 Hybrid Transform Coding 

Various techniques have been developed to process the images with 
highly correlated picture elements to generate, a set of nearly uncor- 
related data. Among them, the DPCM system which was discussed in 
section 5.3 and the transform coding (63 , 19 ] and other schemes are 


Table 5*5 The Performance of Mod ified DPCM W I th Variable 
Length Code (Cj code) 

rms Skipping 

Scheme N R Error Factor 

DB I 3 bits 6.899^* 10.3923 VI 

DB 2 2 bits 7.1580 il.OOl Vi 

DB 3 3 bits S.U'*! 10.1332 2/1 

DB 1» 4 bits 4.2939 9.099 2/1 











Figure 5.24 Reconstructed Picture From DB 4 Compression 
Scheme 


in 


known to be very efficient. Studies on the DPCM and the transform cod" 
Ing had indicated that each scheme has its own advantage and some 
limitations. For example, the transform coding provides excellent per- 
formance at lower bit rate, the coding degradation resulted from this 
scheme is less objectionable to the human vision system and less sensi- 
tive to the channel noise. On the other hand, the DPCM system achieves 
a better coding performance when operated at a higher bit rate, and it 
requires the minimum equipment complexity. Furthermore, the memory or 
buffer size needed for DPCM is very small. The disadvantage of the 
DPCM system is its sensitivity to the picture statistics while the non- 
linear quantizing and variable length code are concerned, in addition to 
its error propagation when the channel noise is introduced during the 
transmission. 

Hybrid coding was first introduced by A. Habibi [64] to combine 
the transform coding v;ith DPCM system to absorb the attractive features 
of both schemes and avoid the limitations of each system. 

Suppose that a picture is an array of MxN picture elements. V/e 
divide each scan line of picture elements into sub-blocks. Each block 
consists of n picture elements v/here N is an integer multiple of n. 

Each of these blocks was processed by the cosine transformation to obtain 
n coefficients. 

Let the picture elements of each sub-block of a scan line be denoted 
as U(jl), where Z = 0,...,n-l. Then, the cosine transform of U(«.) is 
defined as 

yv n-1 

C(0) = — E U(£) 


(5.23) 
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C(K) = ~ Z U()t) cos K=l,2,..«n"l (5.24) 

" £=0 2n 

where C(K) is the Kth cosine transform coefficiento Eq„ (5.24) can be 
rewritten as: 

C(K) = iRe ' U(;i) (5.25) 

where W = i = >T 

and U(s,) = 0 for A = n, n+1,<,..2n“l 

Note that Re {•} represents the real part of the term enclosed. From 
Eq, ( 5 . 25 ), it is seen that all n transform coefficients can be obtained 
by fast Fourier transform (FFT), which reduces the computation efforts 
and cpu time. 

The picture elements within each sub-block are highly correlated 
with the picture elements in the corresponding sub-block on the next 
scan line. We can expect that the coefficients generated by the cosine 
transform to be also highly correlated, because we do the same trans- 
formations to all the sub-blocks. A DPCM system Is used to operate on 
each column of the transformed coefficients to remove the redundancy in 
the vertical direction. 

To encode the coefficient differences more efficiently, we use the 
block quantization which assigns a different number of bits to encode 
the different coefficient differences based on the statistics of the 
coefficient differences. In short, the cosine transform is applied In 
the horizontal direction followed by the DPCM in the vertical direction. 

Some computer simulation results using the hybrid transform coding 
are presented in Figs, 5.25 through 5.26, corresponding to the following 


schemes: 



5.25 Reconstructed Picture From H 1 Compression 
Scheme 







Figure 5.26 Reconstructed Picture From H 2 Compression 
Scheme 


i 

J 1 V. ■ . i ' , v: > ‘ 
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HI; The bit allocation is 1 bit per picture element in average. 

The data compression ratio is 6. 

H2: The bit allocation is 1.5 bits per picture element in average. 

The data compression ratio is 4. 

The bit allocation for the coefficient differences is shown in 
Table 5.6. The performance of the hybrid schemes is shown in Table 5.7. 

It Is obvious that the distortion is not objectionable to the human 
observer, and the degradation is very smooth throughout the whole picture. 
There is no artificial square boundaries in the reconstructed pictures. 
However, this scneme is e little time consuming due to many operations 
required. The time delay of the hybrid technique should be taken into 
account to see whether the time delay can be tolerated. 

5.5 Linear Interpolation Technique 

We recall that the picture elements of a digitized image are highly 
correlated with the neighboring picture elements. The probability that 
a picture element is of the same grey level as its neighbors is rather 
high, A simple and easy to operate data compression technique is to 
take advantage of this picture element correlation. The key idea is to 
skip some picture elements uniformly to provide a proper skip factor. 

First of ail, we divide the entire picture Into blocks of size nxm, 
and transmit only the top left picture element of each block with 6 bits/ 

pel. Thus, a data compression ratio of nxm is achieved (which is the 

same as the skipping factor in this case). As we know that the satellite 

images were oversampled in the ratio of 1,5 Jl» the magnitude of n or m 

should not exceed 2, otherwise the rms error of the reconstructed 
picture will become more significant. Our computer simulation also 
verified this argument. 
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Two data compression techniques based on the linear interpolation 
are designed to provide compression ratios of ^ and 6 respectively. 

L 1; The picture is divided into blocks, Each block is a ?.x2 
array of picture elements. Only the top left picture 
elements of each block are processed and transmitted vvith 
6 bits/pei. Therefore, a data compression ratio of k is 
obtained. The reconstruction algorithm viould be well 
explained through the help of the following example; 

A B C D 

E F G H 

I J K L 

Suppose that A, C, I, and K were the transmitted picture 
elements. The algorithm is to calculate the skipped 
picture elements by the following equations: 


B = 

1 

2 

(A+C) 

J = 

1 

2 

(l+K) 

E = 

1 

2 

(A+l) 

G = 

1 

2 

(C+K) 

F = 

1 

(b+e+g+j) 


The procedure is repeated for all the blocks of the 
entire pictures. 
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L 2: Same as L 1 except the block size is a 2x3 array of picture 
elements. The data compression ratio is 6, The reconstruc- 
tion algorithm is as follows; 

tn the example, if A, D, I, and L were the transmitted picture 
elements, the skipped picture elements can be found through the 

following calculations: 

2 1 

B - J A + y D 

1 2 

C=jA+yD 

1 ? 

K = y I + y L 

E=j(A+l) 


(5.27) 


H » 1 (D+L) 

r 2-^1,, 
r = y E + y H 


G * i E + y H 


The procedure is repeated for all the blocks of the entire 
picture. 

We present the reconstructed pictures of Fig, 5.2 based on schemes 
LI and L 2 in Figs. 5,27 and 5.28, Their performances were shown in 
Table 5.8. 

Since only simple operations were involved in the linear interpo- 
lation technique, the equipment complexity for this technique is mini- 
mized. In fact, a switching circuit device can be used to serve as a 
sample selector to provide appropriate skipping factor. 














As shown In Fig. 5-23, a block diagram of the multi-rate data 
transmission system will operate at 120, 20, and 30 mega bits per second. 
Hero the sample selector played an important role in this system. Note 
that certain spectral band may have poor signal to noise performance. 

For example, channel A of LANOSAT (previously called ERTS) images has 
poor signal to noise performance, thus, 6-bit log scale quantizer is 
bypassed. 

In fact, this is a very efficient and low cost technique, which 
provides reasonably good performance. Time delays and computational 
efforts are minimized. However, the rms error in the reconstructed 
pictures by this technique is slightly higher, especially in L 2, The 
overall performance of the linear interpolation scheme make it a oromls- 
Ing candidate of a multi-rate data transmission system. 







CHAPTER 6 


CONCLUSION 


6,1 Summary of Results 

This work was primarily concerned with the Investigation of an 
efficient, almost real time, data compression system for multi spectral 
satellite pictures and for binary or multi-level pictures obtained from 
the classification of satellite picture. Due to the different character 
istics of these pictures, our Investigation was divided into two parts. 

The first part of this research dealt with efficient data com- 
pression schemes for binary and multi-level pictures. We have modified 
the predictive differential quantizing (PDQ) method to ease the design 
of the compressed data format. Furthermore, an Improved data compres- 
sion scheme called double delta coding (DDC) was Introduced. DDC was 
shown, by computer simulation, to be more efficient than PDQ for those 
pictures where PDQ was better than run length coding (RLC). Both DDC 
and PDQ can be Improved by applying background skipping technique. The 
simulation results showed that the application of background skipping 
on PDQ and DDC significantly reduced the processing time and achieved 
higher data compression. 

The derived data, which Is defined to be the output of the re- 
dundancy remover, should be encoded by the appropriate source codes be- 
fore being transmitted over the channel. We have Investigated the 
prefix and non-prefix source codes. Both of these are variable length 


codes that can take advantage of the nonuniform distribution of the 
derived data. The extension code was formulated by combining the pre- 
fix code and the extension blocks. This class of codes operated rather 
efficiently, and required very little storage space. Moreover, the 
size of the extension block can be adjusted based on the practical 
situation. The code word of the non-prefix code consists of blocks of 
fixed length binary digits. 

In general, the complexity of the objects In the picture played an 
important role in data compression. The pictures taken during the 
flooding period are more complicated than those taken In normal seasons. 
To facilitate the design of the data compression system, we can pre- 
determine the coding scheme for each type of picture. Our simulations 
showed that run length coding Is more efficient than others for those 
flooded pictures; hov;ever, double delta coding Is the most efficient 
scheme for pictures taken during normal seasons. As the complexity of 
the picture and the number of grey levels are increased, none of the 
error-free coding schemes we have Investigated can provide efficient 
data compression. 

The second part of this work Is concerned with data compression 
techniques v/hich would provide data compression ratios of four and six 
for the continuous- tone satellite pictures. Since the complexity of 
the satellite pictures are usually very high, any error free compression 
technique can not guarantee the desired data compression ratios. Thus, 
a certain amount of distortion resulting from the data compression is 
inevitable. To evaluate the performance of the compression schemes, the 
root mean square error was used as the error criterion. 
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The DPCM system Is a very efficient data compression system with 
minimal design coniplexity and high operation speed# We have Investi- 
gated the modified DPCM system which used a sample selector to skip 
some picture elements. To keep the rms error as small as possible, we 
found that it v/as better to limit the maximal skipping factor to two 
ifi each direction. With fixed length codes, the uniform quantizer will 
be easier and faster to operate. On the other hand, the nonuniform 
quantizer will provide better performance when the variable length code 
is employed. Furthermore, the variable length code enables us to 
choose a quantizer with more quantization levels to achieve smaller rms 
error. 

The hybrid transform coding system, which combines the DPCM system 
with the transformation technique, takes advantage of both schemes. 

The reconstructed picture of the hybrid scheme is less objectionable to 
human observers and the resulting rms error was minimal among all the 
compression schemes we have investigated. However, it is the most time 
consuming scheme and has the highest cost of implementation. 

The linear interpolation technique is the simplest compression 
scheme both in implementation and processing. For the satellite pic- 
tures which were oversampled at the ratio of l.Ssl* a skipping factor 
of two in each direction is reasonable. Thus, the distortion resulting 
from the L2 scheme with a skipping factor of six is more significant 
than the LI scheme with a skipping factor of four. In general, the 
linear interpolation technique is an efficient and low cost compression 
scheme whose performance is reasonably good. 
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In conclusion^ the selection of a compression scheme for a multi- 
rate data transmission system depends on several factors such as the 
rms error, the computational cost, time delay, and the complexity of 
implementation. 

6,2 Suggestion for Future Research 
The compressed data format Is designed to pack the encoded derived 
data into a compact format and to enable the receiver to Identify the 
data string for each scan line such that the number of objects In each 
scan line can be decoded correctly. Hov^ever, the compressed data 
format did not offer the protection and error detection capability to 

each individual encoded Index (I.e., A's or run lengths). When the 

) 

channel is noisy and the transmitted Indices are corrupted, the objects 
in each scan line will be displaced causing a degradation in the recon- 
structed picture. 

One possible way to solve this problem Is to perform channel cod- 
ing before the encoded data are transmitted such that the errors occur- 
ring in the encoded indices can be detected at the receiver. However, 
it will decrease the data compression ratio. Another possible way is 
to investigate an encoder which can be used as both source and channel 
coder. Some researchers have reached preliminary results in this open 
area of research; however, those methods are very limited and can not 
be applied to practical systems. 
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APPENDIX 

Smith's Nonuniform Quantization 

When the probability density function of the signal can be approxi- 
mated by the exponential density function 

p(e) « -^exp (- lei) 

2/a a 

where a is the variance of the signal e 

The function f(e) Is formed such that when e takes some value in 
the range of V to -V, f assumes a proper value. 

f(e) = - ^ «,n [1 ■ (1 ■ exp (-m))] 

f (e) = f(-e) 

where m = /z \'/3o for V = 63, a = 12,18, we present a list of 

tables as follows: 




